EDI-GRAPHIC: A TOOL TO STUDY PARAMETER DISCRIMINATION AND CONFIRM IDENTIFIABILITY IN BLACK-BOX MODELS, AND TO SELECT DATA-GENERATING MACHINES

SummaryIn a Data-Generating Experiment (DGE), the data, X, is often obtained from Black-Box and approximated with learning machine/sampler, f(Y,θ);θ∈Θ, Y random, f known. When X has unknown c.d.f., Fθ, non-identifiability of θ cannot be confirmed may limit predictive accuracy learned model, f(Y,θ̂);θ̂ estimate θ. Using properties Expected P-value for Kolmogorov-Smirnov test, Empirical Discrimination Index (EDI) Proportion P-Values (PPVI) are introduced: i) to confirm almost surely, discrimination θ*, i.e. Fθ≠Fθ*, ii) EDI-graphics identifiability θ(∈Θ) by repeating θ* in fine sieve Θ, iii) compare PPVIs DGEs select use DGE greater parameter smaller number violating Among applications, EDI PPVI explain why g-estimate Tukey’s g-and-h model better than that g-and-k unless sample size extremely large; h=h0=k. indicate Normal machines have Sigmoid their parameters non-identifiable.